Validity Theory Overview

Principles

Definition

Validity theory is an academic framework that provides principles and methods for evaluating the appropriateness, meaningfulness, and utility of the interpretations and uses of data or assessment results. It investigates the comprehensive body of evidence required to support such claims, emphasizing that validity is an ongoing process of building and scrutinizing a validity argument.

Ontological type

Validity Evidence Types

Argument-Based Approaches

Consequences of Use

Key developments

Key Figures

Evidence Integration Era era

Samuel Messick [1] was a central figure in validity theory during the Evidence Integration Era, with affiliations including the University of Pennsylvania [3] and The Graduate Center, CUNY [4]. Messick [1] highlighted the meaning of scores and the ethics of assessment, as discussed in Meaning and Values in Test Validation: The Science and Ethics of Assessment [8], and in Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning [7], arguing these contributions were essential for formalizing cross-context validity in this era. Neal Schmitt [2] is associated with the University of Tennessee at Knoxville [5] and Michigan State University [6] during this era. Schmitt [2] conducted METAANALYSES OF VALIDITY STUDIES PUBLISHED BETWEEN 1964 AND 1982 AND THE INVESTIGATION OF STUDY CHARACTERISTICS [9], and raised FORTY QUESTIONS ABOUT VALIDITY GENERALIZATION AND META-ANALYSIS [10], shaping contemporary approaches to validity generalization and meta-analysis and explaining why context matters for generalized validity claims.

Argument and Policy Validity era

Michael T. Kane[1] is a leading figure in validity theory whose work in the Argument and Policy Validity era (1996–2013) relates to affiliations at the University of Wisconsin–Madison[3] and the University of Utah[4]. His key contributions during this era include clarifying how interpretations of test scores are validated and identifying threats to the valid use of assessments, as seen in Threats to the Valid Use of Assessments[9], Current Concerns in Validity Theory[8], and Validating the Interpretations and Uses of Test Scores[7], which together advanced credibility, transferability, dependability, and stakeholder-relevant outcomes in policy contexts. Klaas Sijtsma[2] is another central figure in this era, working across Utrecht University[5] and the University of Twente[6]. His principal contribution is critiquing overreliance on Cronbach’s Alpha, notably in the 2008 paper On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha[10], which spurred renewed attention to measurement reliability and validity within policy-oriented validation debates.

Use-Centered Validity era

Richard J. Shavelson [1] is associated with Stanford University [3] and the University of Colorado Boulder [4] during this era. In Beyond Dichotomies [7], the work reframes validity beyond simple dichotomies by foregrounding score use in decision contexts, a shift that underpins use-centered validity. Eiko I. Fried [2] is tied to Icahn School of Medicine at Mount Sinai [5] and Yale University [6] during this period. Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them [8] critiques questionable measurement practices and offers concrete guidance to promote methodological transparency and instrument-level safeguards, essential for ensuring scores are used responsibly in use-centered validity.